REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  NO.  0704-0188 


The  public  reporting  burden  for  this  coilection  of  information  is  estimated  to  average  1  hour  per  response,  inciuding  the  time  for  reviewing  instructions, 
searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  compieting  and  reviewing  the  coiiection  of  information.  Send  comments 
regarding  this  burden  estimate  or  any  other  aspect  of  this  coilection  of  information,  including  suggesstions  for  reducing  this  burden,  to  Washington 
Headquarters  Services,  Directorate  for  information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Ariington  VA,  22202-4302. 
Respondents  shouid  be  aware  that  notwithstanding  any  other  provision  of  iaw,  no  person  shaii  be  subject  to  any  oenaity  for  failing  to  comply  with  a  coiiection 
of  information  if  it  does  not  dispiay  a  currentiy  vaiid  OMB  controi  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


2.  REPORT  TYPE 
Conference  Proceeding 


1.  REPORT  DATE  (DD-MM-YYYY) 
30-09-2014 


4.  TITLE  AND  SUBTITLE 

Assessemnt  of  cognitive  compoinents  of  decision  making  with 
military  versions  of  the  IGT  and  WCST 


3.  DATES  COVERED  (From  -  To) 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 


6.  AUTHORS 

Quinn  Kennedy,  Pete  Nesbitt,  Jon  Alt 


5c.  PROGRAM  ELEMENT  NUMBER 
611102 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAMES  AND  ADDRESSES 

Naval  Postgraduate  School  (NPS-Monterey)  14,973.00 

1  University  Circle 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


Monterey,  CA  93943  -5000 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS 
(ES) 

U.S.  Army  Research  Office 
P.O.  Box  12211 

Research  Triangle  Park,  NC  27709-2211 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 
ARO 


11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

62626-MA.3 


12.  DISTRIBUTION  AVAILIBILITY  STATEMENT 
Approved  for  public  release;  distribution  is  unlimited. 


13.  SUPPLEMENTARY  NOTES 

The  views,  opinions  and/or  findings  contained  in  this  report  are  those  of  the  author(s)  and  should  not  contrued  as  an  official  Department 
of  the  Army  position,  policy  or  decision,  unless  so  designated  by  other  documentation. 


14.  ABSTRACT 

The  U.S.  Army  is  focused  on  improving  leader  decision  making,  yet  little  is  known  as  to  how  military  officers 
develop  optimal  decision  making.  Two  key  components  of  optimal  decision  making  are  reinforcement  learning  and 
cognitive  flexibility.  Thirty-four  military  officers  completed  military  versions  of  two  standard  cognitive 
assessments,  the  Iowa  Gambling  Test  and  the  Wisconsin  Card  Sorting  Task,  while  their  eye  movements  were 
tracked.  Results  indicated  that  the  military  versions  of  these  tasks  successfully  provided  objective  assessments  of 

^  4-  1  y-V  -t  /“I  y-fc  y-V  4- IT  T  y-V  42|  1  .«  4-T  T  I  J««y-V  1  -t  -t  T  yV  1  -<•  r  y-1  y-V  i-1  y-V  y-V  T  r  y-V  y-V  -»  T  y-V  y-V  4-y-1  y-VT  T .«  /“I  y-V  -t  y-1  4"  y-1  4-  y-V  4-T'%  y-V 


15.  SUBJECT  TERMS 

Decision  making  ,  wargames,  Iowa  Gambling  Test,  Wisconsin  Card  Sorting  Test 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 

15.  NUMBER 

a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

ABSTRACT 

OF  PAGES 

UU 

UU 

UU 

UU 

Ouinn  Kennedv 


19b.  TELEPHONE  NUMBER 
831-656-2618 


Standard  Form  298  (Rev  8/98) 
Prescribed  by  ANSI  Std.  Z39. 18 


Report  Title 

Assessemnt  of  cognitive  compoinents  of  decision  making  with  military  versions  of  the  IGT  and  WCST 

ABSTRACT 

The  U.S.  Army  is  focused  on  improving  leader  deeision  making,  yet  little  is  known  as  to  how  military  offieers 
develop  optimal  deeision  making.  Two  key  components  of  optimal  deeision  making  are  reinforeement  learning  and 
cognitive  flexibility.  Thirty-four  military  officers  completed  military  versions  of  two  standard  cognitive  assessments, 
the  Iowa  Gambling  Test  and  the  Wisconsin  Card  Sorting  Task,  while  their  eye  movements  were  tracked.  Results 
indieated  that  the  military  versions  of  these  tasks  sueeessfully  provided  objective  assessments  of  reinforeement 
learning  and  eognitive  flexibility.  Preliminary  analyses  of  eye  movements  provide  insights  into  the  subjects’  decision 
strategies.  Training  implieations  of  the  results  are  discussed. 

Conference  Name:  Human  Factors  and  Ergonomics  Society  Annual  Conference 
Conference  Date:  October  29,  2014 


ASSESSMENT  OF  COGNITIVE  COMPONENTS  OF  DECISION 
MAKING  WITH  MILITARY  VERSIONS  OF  THE  IGT  AND  WCST 


Quinn  Kennedy,  Peter  Nesbitt,  and  Jon  Alt 
Naval  Postgraduate  School 

The  U.S.  Army  is  focused  on  improving  leader  decision  making,  yet  little  is  known  as  to  how  military 
officers  develop  optimal  decision  making.  Two  key  components  of  optimal  decision  making  are 
reinforcement  learning  and  cognitive  flexibility.  Thirty-four  military  officers  completed  military  versions 
of  two  standard  cognitive  assessments,  the  Iowa  Gambling  Test  and  the  Wisconsin  Card  Sorting  Task, 
while  their  eye  movements  were  tracked.  Results  indicated  that  the  military  versions  of  these  tasks 
successfully  provided  objective  assessments  of  reinforcement  learning  and  cognitive  flexibility. 
Preliminary  analyses  of  eye  movements  provide  insights  into  the  subjects’  decision  strategies.  Training 
implications  of  the  results  are  discussed. 


INTRODUCTION 

As  the  U.S.  Army  focuses  on  enhancing  leader 
development  and  decision-making  to  improve  the 
effectiveness  of  forces  in  combat,  the  importance  of 
understanding  how  to  effectively  train  decision-makers  and 
how  experienced  decision-makers  arrive  at  optimal  or  near 
optimal  decisions  has  increased  (Lopez,  2011).  Two  cognitive 
characteristics  necessary  for  military  personnel  to  reach 
optimal  decision  making  are  reinforcement  learning,  the 
ability  to  learn  from  trial  and  error,  and  cognitive  flexibility, 
the  ability  to  recognize  when  the  rules  have  changed  or  that 
the  current  strategy  no  longer  works  (Vartanian  &  Mande., 
2011).  Although  many  laboratory  tests  of  reinforcement 
learning  and  cognitive  flexibility  exist,  these  tasks  may  not 
necessarily  capture  military  decision  making  due  to  the  high 
stakes  and  uncertain  environment  in  which  military  decisions 
are  made.  Assessment  tools  that  leverage  wargames, 
simulations  of  realistic  military  scenarios,  to  evaluate  these 
two  cognitive  characteristics  are  needed.  We  determined  that 
two  common  psychological  tests  that  measure  reinforcement 
learning  and  cognitive  flexibility,  the  Iowa  Gambling  Task 
(IGT)  (Bechara  et  ah,  1994)  and  the  Wisconsin  Card  Sorting 
Task  (WCST)  (Grant  &  Berg,  1948)  could  be  modified  to 
provide  a  more  realistic  military  context  as  a  first  step  towards 
understanding  military  decision  making. 

The  IGT  was  developed  to  measure  prefrontal 
damage  (Bechara  et  al.  1994).  Persons  with  prefrontal  damage 
tend  to  have  difficulty  detecting  the  long-term  consequences 
of  their  decisions  and  actions.  In  this  task,  subjects  receive  a 
loan  of  $2,000  of  play  money  and  are  asked  to  make  a  series 
of  decisions  to  maximize  the  profit  on  the  loan.  Each  decision 
entails  selecting  one  card  at  a  time  from  any  of  four  available 
decks  of  cards  (decks  A-D).  All  cards  give  money  and  some 
cards  also  issue  a  penalty.  Decks  differ  in  the  amount  of 
money  given  on  a  single  trial  ($50  or  $100)  as  well  as  the 
frequency  and  severity  of  penalties  ($0  to  $1250).  Healthy 
subjects  should  learn  through  reinforcement  learning  which 
decks  have  the  best  long  term  payoffs  (decks  C  and  D) 
(Bechara  et  al.,  1994;  Steingroever  et  al.,  2013).  Main 
measures  of  decision  performance  are  total  money  won  and  an 


advantageous  selection  bias  (the  proportion  of  good  decks 
selected  minus  the  proportion  of  bad  decks  selected). 

The  WCST  taps  the  working  memory,  shifting  and 
inhibition  components  of  executive  function  (Grant  &  Berg, 
1948).  Subjects  view  five  cards,  one  card  displayed  at  the  top 
center  of  the  screen,  the  remaining  four  displayed  across  the 
bottom  of  the  screen.  Each  card  contains  symbols  that  vary  in 
number,  shape,  and  color.  Over  several  trials,  participants  try 
to  figure  out  the  matching  rule  that  will  correctly  match  the 
card  on  the  top  of  the  screen  with  one  of  the  four  cards  at  the 
bottom  of  the  screen.  Unbeknownst  to  subjects,  the  matching 
rule  changes  once  they  have  10  consecutive  correct  matches. 
For  example,  after  10  consecutive  correct  matches  based  on 
the  color  of  the  symbols,  the  matching  rule  could  then  change 
to  the  number  or  shape  of  the  symbols.  Thus,  subjects  must 
not  only  learn  and  maintain  in  working  memory  the  correct 
matching  rule  while  inhibiting  irrelevant  stimuli,  but  also 
exhibit  cognitive  flexibility  in  detecting  when  the  rule  has 
changed  (Grant  &  Berg,  1948).  The  task  is  complete  when 
subjects  either  successfully  complete  two  rounds  of  each 
matching  rule  or  128  trials.  Main  performance  measures 
include  total  percent  correct,  percent  of  perseverative 
responses  (the  number  of  incorrect  responses  that  would  have 
been  correct  for  the  previous  matching  rule),  the  number  of 
matching  rules  achieved  and  total  number  of  trials  completed 
(fewer  indicates  better  performance). 

The  purpose  of  this  study  was  to  modify  two  existing 
cognitive  assessments  that  measured  reinforcement  learning 
and  cognitive  flexibility  in  order  to  assess  active  duty  military 
officers’  decision  making  behavior  on  these  tasks.  The 
convoy  task,  in  which  subjects  incur  or  receive  enemy  or 
friendly  damage  is  analogous  to  the  IGT,  whereas  the  map 
task  is  modified  from  the  WCST.  In  order  to  gain  further 
insight  into  how  military  decision  makers  value  information, 
eye  tracking  data  was  captured  for  each  subject  during  each 
task.  Numerous  studies  indicate  that  eye  movement  data  via 
eye  tracking  technology  can  provide  valuable  insights  into 
subjects’  attention  allocation  patterns  and  underlying  cognitive 
strategies  during  real-world  tasks  (Kasarskis  et  al.,  2001; 
Marshall,  2007;  Sullivan  et  al.,  2011).  We  predicted: 


1)  Convoy  task:  Subjects  will  demonstrate  reinforcement 
learning  by  having  a  positive  advantageous  selection  bias,  and 
by  correctly  reporting  which  routes  are  the  safest  and  most 
dangerous. 

2)  Map  task:  Subjects  will  demonstrate  cognitive  flexibility 
by  having  low  rates  of  perseverative  responses,  completing  at 
least  3  matching  imles,  and  having  at  least  70%  correct  trials. 

3)  Exploratory  analyses  from  the  eyetracking  data  will  provide 
insights  into  subjects'  prioritization  of  information. 

METHOD 

Participants 

The  study  collected  data  from  34  military  officers  from  all 
branches  of  service:  9  U.S.  Army,  11  U.S.M.C.,  10  U.S. 
Navy,  3  U.S.  Coast  Guard,  and  1  U.S.  Air  Force.  Mean  age 
was  35.11  years  (SD  4.9)  with  a  mean  time  in  service  of  12.7 
years  (SD  4.42)  of  which  the  average  time  deployed  was  19.57 
months  (SD  12.12)  (note  one  subject  did  not  report  their 
deployment  time).  Of  the  31  subjects  with  deployment 
experience,  the  mean  time  since  their  last  deployment  was 
37.98  months  (SD  25.18)  and  19  of  those  deployments  were  to 
ground  combat  zones  (Iraq  or  Afghanistan).  A  majority  of  the 
subjects,  n=24,  served  as  staff  officers  during  their  most  recent 
deployment.  The  majority  of  the  subjects  were  male  (30 
males,  4  females)  and  the  majority  of  subjects  possessed  20/20 
or  better  visual  acuity  (n=29). 

Measures 

Convoy  task  (Modified  IGT).  On  a  computer 
monitor,  subjects  see  four  identical  routes  (Figure  1).  They 
are  instructed  that  over  several  trials,  they  must  decide  on 
which  route  to  send  convoys  and  based  on  each  decision,  they 
may  incur  enemy  damage  (good  outcome)  or  receive  friendly 
damage  (bad  outcome).  They  also  are  instructed  that  the 
pictures  of  the  routes  are  identical.  Their  goal  is  to  achieve  the 
highest  possible  total  damage  score  by  maximizing  enemy 
damage  and  minimizing  the  friendly  damage  accrued  over  all 
trials.  These  routes  have  the  same  payout  format  as  the  decks 
of  the  original  IGT  (see  Table  1):  routes  3  and  4  are 
considered  good;  routes  1  and  2  are  considered  bad.  Subjects 
receive  immediate  feedback  on  each  trial  on  the  current  total 
damage  score,  and  enemy  damage  and  friendly  damage  that 
occurred  on  that  trial.  Based  on  pilot  data,  we  extended  the 
number  of  trials  from  100  (as  in  the  IGT)  to  200,  as  pilot 
subjects  needed  more  than  100  trials  to  detect  the  long-term 
payout.  Decision  performance  variables  are  measured  using 
typical  IGT  variables:  total  damage  score,  frequency  with 
which  each  route  was  selected,  an  advantageous  selection  bias 
(proportion  of  good  routes  selected  minus  the  proportion  of 
bad  routes  selected). 


Select  route  for  next  convoy. 


100  -250 


Figure  1  Screenshot  of  convoy  task.  Route  1  is  the  top  left  route;  route  3 
is  the  bottom  left  route.  On  this  particular  trial,  the  subject  selected  route  1 
and  received  100  enemy  damage  and  -  250  friendly  damage. 


Table  i  Amount  of  enemy  damage  accrued  on  each  trial  by  route  and 
script  of  the  payout  schedule  for  fi-iendly  damage.  Here,  on  the  2"^  time  that  a 
subject  selected  Route  1,  they  would  receive  -250  in  friendly  damage. 


Route  1 

Route  2 

Route  3 

Route  4 

Enemy 

damage 

100 

100 

50 

50 

Friendly 

damage 

Selection 

1 

-350 

0 

-50 

-250 

2 

-250 

-1250 

-50 

0 

3 

0 

0 

0 

0 

4 

-200 

0 

-50 

0 

5 

0 

0 

0 

0 

6 

-300 

0 

-50 

0 

7 

0 

0 

0 

0 

8 

-150 

0 

-50 

0 

9 

0 

0 

0 

0 

10 

0 

0 

0 

0 

11 

-350 

0 

-50 

-250 

Map  task  (Modified  WCST).  On  a  computer  screen, 
subjects  see  five  maps:  one  map  is  at  the  top  center  of  the 
screen;  the  remaining  four  are  across  the  bottom  of  the  screen. 
Each  map  contains  military  graphic  control  symbols  that  vary 
in  meaning,  color  and  shape  (FM  1-02,  Operational  Terms  and 
Graphics,  United  States,  2004).  The  symbols  fall  into  three 
different  categories:  friendly  force  (blue  symbols),  type  of 
intended  action,  such  as  ambush  (black  symbols),  and  type  of 
enemy  force  (red  symbols).  Within  each  category,  there  are 
four  different  possible  symbols,  each  indicating  a  particular 
type  of  friendly  force,  intended  action,  or  enemy  force. 
Subjects  are  instructed  to  match  one  of  the  four  lower  maps  to 
the  top  one  over  an  unknown  number  of  trials.  As  in  the 
original  WCST,  the  subject  is  not  told  that  the  matching  rule 
will  change  following  10  consecutive  correct  matches.  The 
task  is  completed  when  either  the  subject  has  successfully 
completed  two  rounds  of  each  matching  rule  for  a  total  of  six 
rounds  or  until  they  have  completed  128  trials.  One  small 
modification  from  the  WCST  is  that  not  all  maps  have  all 


three  types  of  symbols  and  subjects  can  match  maps  based  on 
the  absence  of  a  symbol  (see  Figure  2).  Decision  performance 
variables  are  measured  using  typical  WCST  variables:  Total 
number  of  trials  (fewer  indicates  better  performance),  percent 
correct,  percent  perseverative  responses  (percent  of  trials  in 
which  subjects  incorrectly  used  the  same  matching  rule  as  in 
their  previous  selection),  number  of  trials  to  complete  the  first 
matching  rule,  and  number  of  rules  achieved  (max  number  = 
6) 


Figure  2  Screenshot  of  a  trial  in  the  map  task.  On  this  trial,  the  subject 
should  sort  on  intended  action  graphics  (black),  and  therefore  should  seleet 
the  rightmost  map. 


Demographics  Survey^  Demographic  information 
regarding  age,  gender,  service  branch,  rank,  and  deployment 
experience  were  captured  in  the  demographic  survey. 

Post-task  survey.  The  post-task  survey  included 
questions  regarding  a  ranking  of  the  routes  in  the  convoy  task 
from  safest  to  most  dangerous,  decision  strategies,  and  how 
quickly  they  noticed  that  the  matching  rule  had  changed  in  the 
map  task. 

Visual  acuity.  The  Snellen  eye  chart  was 
administered  to  verify  that  all  subjects  had  adequate  visual 
acuity  to  see  the  icons  and  text  on  the  computer  screen. 

Eyetracking.  Subjects’  eye  movements  were 
measured  while  they  completed  the  tasks  with  the  use  of  the 
FaceLAB  eyetracking  equipment.  Eye  movements  were 
measured  in  number  of  fixations  in  each  ROIs  and  percent  of 
time  subjects  looked  at  each  ROT 

Equipment 

The  equipment  used  in  this  study  consisted  of  a  laptop 
computer,  two  eye  tracking  stereo  cameras,  and  a  desktop 
computer.  The  laptop  ran  FaceLAB  5.0.7  30  software  on  a 
Windows  XP  operating  system.  The  stereo  cameras  supplied 
data  to  FaceLAB  on  the  laptop.  The  desktop  computer  ran  the 
EyeWorks  data  collection  suite  on  the  Windows  7  operating 
system.  The  stereo  cameras  used  12  mm  lenses  to  detect 
infrared  light  reflected  off  the  subjects'  eyes  and  face  to 
monitor  the  position  of  the  head  and  direction  of  the  eye  gaze. 
This  data  was  fed  from  the  laptop  to  the  EyeWorks  Record 
software  on  the  desktop. 


Procedures 

This  study  was  approved  by  the  institution’s  IRB.  Subjects 
first  completed  the  approved  consent  form,  followed  by  the 
demographic  survey,  visual  acuity  test  and  eye  tracking 
calibration.  Once  all  calibration  steps  were  satisfied,  subjects 
completed  the  convoy  task  and  then  the  map  task  while  their 
eye  gaze  was  monitored.  Finally,  subjects  answered  the  post 
task  survey  questions. 


RESULTS 


Convoy  task 

All  analyses  utilized  a  two  tailed  .05  alpha  level. 
Although  mean  total  damage  score  was  above  2000  and  the 
advantageous  selection  bias  was  positive,  results  were  not 
significant  (p's  >.05)  (see  Table  2).  As  would  be  expected, 
total  damage  score  was  negatively  correlated  with  the  number 
of  high  friendly  damage,  (r  =  -.87,  p  <  .001),  as  well  as 
frequency  of  friendly  damage  (r  =.39,  p  <  .05).  Officers  also 
successfully  distinguished  between  safe  and  dangerous  roads, 
2 

(X  (3)  =  23.63,  p  =  .005).  In  a  question  asking  subjects  to 
rank  order  the  routes  from  safest  to  most  dangerous,  42% 
reported  route  4  as  the  safest  followed  by  route  3  (27%), 
whereas  42%  of  subjects  reported  route  1  as  the  most 
dangerous,  followed  by  route  2  (33%). 

Table  2  reveals  that  subjects  benefited  from  having 
200  trials  instead  of  100.  Results  from  paired  f-tests  indicated 
that  the  advantageous  selection  bias  improved  in  trials  101  - 
200  compared  to  trials  1-100  (t(33)  =  2.87,  p  =  .007)  and  a 
trend  for  people  to  learn  to  avoid  high  friendly  damage  (t(33) 
=  1.85,  p  =.07)  in  the  second  half  of  the  wargame. 
Improvements  in  decision  performance  were  due  to  the 
decrease  in  route  2  selection  (t(33)=2.70,  p  =  .01),  and 
increase  in  route  3  selection  (t(33)  =  1.87,  p  =  .07). 
Improvements  in  decision  performance  over  time  are  captured 
in  Figure  3,  which  indicates  that  only  after  about  trial  130  did 
subjects'  total  damage,  on  average,  exceed  the  baseline  of 
Figure  3  also  illustrates  the  large  range  of  variability  in 
decision  performance. 


Table  2  Descriptive  statistics  of  convoy  task  decision  variables  for  the  1  “ 
100  trials,  trials  101  -  200,  and  all  200  trials 


Performance 

variable 

First  1 00  trials 

Mean  (sd) 

Trials  101-200 

Mean  (sd) 

All  200  trials 

Mean  (sd) 

Total  damage  score 

2077.94 

(883.96) 

N/A 

2402.94 

(1725.69) 

No.  trials  with 
friendly  damage 

24.5  (6.46) 

26.65  (7.44) 

51.15  (11.05) 

No.  trials  with 
heavy  friendly 
damage 

3.62  (1.39) 

3.06(1.72) 

6.68  (2.59) 

Route  selection 
frequency  (%) 

Route  1 

13.82  (7.88) 

12.56  (8.59) 

13.19(7.27) 

Route  2 

38.91  (14.30) 

30.74(16.84) 

34.82  (12.82) 

Route  3 

21.62  (16.59) 

28.77(20.63) 

25.19(15.02) 

Route  4 

25.64  (12.93) 

27.94(18.48) 

26.79(12.39) 

Adv.  selection  bias 

-5.47  (30.73) 

13.41  (41.57) 

7.94  (62.38) 

Figure  3  Mean  total  damage  score  per  trial  (blue  line)  with  95%  Cl  (red 
dotted  line). 

Finally,  preliminary  eyetracking  analyses  revealed  that 
subjects  spent  most  of  the  time  looking  at  the  routes  and  relied 
more  heavily  on  friendly  damage  information  than  enemy  or 
total  damage  (see  Table  3). 

Table  3  Mean  number  of  fixations  and  percent  time  spent  in  each  convoy 
task  ROI. 


ROI 

Mean  (sd).  % 

Enemy  damage 

318.53  (355.09),  1.74% 

Friendly  damage 

1545.66(1678.50),  9.15% 

Total  damage 

141.94  (373.10),  1.03% 

Routes 

15565.41  (8001.44),  88.56% 

Map  task  results 

Results  indicate  that  most  subjects  were  able  to 
determine  the  matching  rules  and  that  the  matching  rule 
changed  periodically.  Total  percent  correct  was  not 
significantly  different  from  70%  (95%  Cl:  59.81%  -  70.58%). 
Subjects  completed  an  average  of  3.21  matching  rules  (95% 
Cl:  2.53  -  3.88).  When  subjects  committed  an  error,  they 
tended  to  be  non-perseverative  errors:  On  average,  non- 
perseverative  errors  occurred  on  33.56%  (sd  =  16.46%)  of  all 
trials,  whereas  perseverative  errors  occurred  on  10%  (sd  = 
8.79%)  of  all  trials.  Four  subjects  never  completed  the  first 
matching  rule.  In  the  post  task  questionnaire,  44%  reported 
that  they  “immediately”  recognized  that  the  matching  rule  had 
changed,  29%  “after  a  few  trials,”  15%  “after  several  trials,” 
and  12%  “did  not  realize  matching  rule  had  changed.”  There 
was  a  positive  correlation  between  how  long  it  took  subjects  to 
realize  that  the  matching  rule  had  changed  and  the  total 
number  of  trials  completed  (r  =  .46,  p  <  .05)  and  a  negative 
correlation  between  this  self-reported  variable  and  percent  of 
correct  trials  (r  =  -.53,  p  <  .05).  Table  4  outlines  subjects’ 
performance  on  the  main  decision  performance  variables. 


Table  4.  Descriptive  statistics  of  map  task  decision  variables. 


Perfonnance  variables 

Mean  fsdf  median,  ranse 

No.  trials 

119.35  (16.52)  128,  76-  128 

%  correct 

65.19  (15.43)  68.75,  36.72-86.25 

No.  Perseverative  responses 

11.82(11.12)9,  0-37 

No.  Non-perseverative  errors 

41.85  (22.52)38,8-81 

No.  trials  to  complete  P'  rule 

42.9(28.95)  34,  14-  121 

(n  =  30) 

No.  categories  achieved 

3.21  (1.94)  4,  0-5 

Failure  to  maintain  set 

2.32  (1.49)  2,  0-5 

Preliminary  eyetracking  results  indicate  that  subjects 
spent  the  majority  of  the  time  looking  at  the  example  card  at 
the  top  of  the  screen,  and  then  appear  to  spend  more  time 
looking  at  the  cards  in  the  center  of  the  screen  (cards  2  and  3) 
than  cards  on  the  farthest  sides  of  the  screen  (cards  1  and  4) 
(see  Table  5). 

Table  5  Mean  number  of  fixations  and  percent  time  spent  in  each  map  task 
ROI. 


ROI 

Mean  (sd),  % 

Example  card 

9289.55  (5372.24),  46.95% 

Card  1 

1360.32(1146.73),  6.12% 

Card  2 

3115.29(2300.53),  14.00% 

Card  3 

4525.81  (3026.41),  21.77% 

Card  4 

2425.58(1434.77),  11.16% 

DISCUSSION 

Overall,  results  indicate  that  the  modified  tasks 
successfully  captured  reinforcement  learning  and  cognitive 
flexibility.  Results  from  the  convoy  task  were  consistent  with 
other  studies  in  which  healthy  adults  completed  the  IGT 
(Steingroever  et  al,  2013).  Although  the  total  damage  score 
and  advantageous  selection  bias  results  were  not  significant, 
subjects  correctly  reported  which  routes  were  safe  and  which 
were  dangerous.  Subjects'  score  on  the  modified  IGT  benefited 
from  the  additional  100  trials  beyond  the  standard  IGT 
protocol.  Subjects’  advantageous  selection  bias  significantly 
increased  due  to  a  shift  in  route  selection  patterns,  potentially 
attributable  to  the  occun'ence  of  reinforcement  learning. 
Additionally,  preliminary  eyetracking  results  indicate  that 
subjects  tended  to  prioritize  information  regarding  friendly 
damage  over  information  regarding  total  damage  and  enemy 
damage  scores  in  making  their  decisions,  highlighting  the 
potential  impact  of  the  military  context.  Also  consistent  with 
previous  studies  of  the  IGT  (Steingroever  et  al  ,  2013),  all 
convoy  measures  showed  large  amounts  of  variability. 


suggesting  that  individual  differences  occur  even  among 
healthy  subjects. 

Results  from  the  map  task  were  somewhat  lower  than 
what  is  typically  found  on  the  WCST  for  healthy  subjects 
(Shan  et  al,  2008).  However,  subjects'  perseverative  response 
rates  were  relatively  low,  indicating  that  errors  were  not  due  to 
lack  of  cognitive  flexibility.  One  reason  that  subjects  may  not 
have  performed  as  well  as  predicted  is  because  subjects' 
military  experience  actually  may  have  made  it  harder  for  them 
to  detect  the  matching  rule.  Unlike  the  original  WCST,  the 
symbols  in  the  map  task  are  meaningful:  each  map  can  be 
'read'  as  a  sentence  by  experienced  military  personnel:  some 
type  of  friendly  force  should  do  an  intended  action  upon  an 
enemy  force.  Thus,  these  experienced  military  officers  may 
have  attempted  to  match  the  maps  based  on  meaning,  rather 
than  simply  on  color  and  shape. 

This  work  holds  several  future  directions  that  can 
inform  and  increase  the  efficiency  of  training  designed  to 
improve  military  decision  making  performance.  As  mentioned 
above,  a  large  range  of  variability  in  decision  performance 
occurred  in  both  tasks.  Thus,  one  future  direction  is  to 
examine  individual  differences  in  decision  performance  and 
prioritization  of  information.  For  example,  do  differences  in 
cognitive  function,  such  as  working  memory  or  visual 
processing  speed,  explain  some  of  the  variability  in  decision 
performance  in  the  military  decision  problems?  Additionally, 
can  eye  movement  patterns  predict  decision  performance? 
Finally,  results  from  the  convoy  task  demonstrate  that  at  some 
point  over  the  course  of  the  200  trials,  subjects  transitioned 
from  exploring  different  routes  to  figuring  out  each  routes' 
long  term  damage  payout  and  exploiting  that  knowledge.  By 
looking  at  decision  and  eye  movement  data  on  a  trial  by  trial 
basis,  can  the  moment  in  which  subjects  start  to  exploit  their 
knowledge  be  pinpointed?  Future  work  will  include 
increasing  the  complexity  of  the  decision  making  setting  by 
analyzing  decision  makers’  performance  in  sequential  decision 
making  tasks  with  delayed  rewards  and  more  noisy  wargame 
settings. 

Summary 

Wargames  are  a  preferred  method  of  training  military 
personnel  to  make  optimal  military  decisions.  However, 
wargames  typically  are  not  assessed  objectively  and  may  not 
focus  on  training  two  cognitive  functions  necessary  for 
optimal  decision  making:  reinforcement  learning  and 

cognitive  fiexibliity.  The  purpose  of  this  study  was  to  take  the 
first  steps  to  bridge  the  gap  between  the  study  of  decision 
making  ability  in  the  field  of  cognitive  psychology  and  the 
study  of  decision  making  in  a  military  setting.  The  use  of  well 
known  objective  assessments  to  assess  the  effectiveness  of 
training  designed  to  improve  reinforcement  learning  and 
cognitive  flexibility  shows  great  potential.  Results 
demonstrate  successful  modification  of  the  IGT  and  WCST 
into  a  military  context.  Future  directions  focus  on  explaining 
individual  differences  in  decision  performance  and  identifying 
the  moment  in  which  subjects'  transition  from  exploration  of 
the  environment  to  exploitation  of  knowledge  obtained  about 
the  environment.  Future  studies  will  examine  military 


decision  making  performance  in  sequential  decision  making 
tasks  with  delayed  rewards  and  more  realistic  military 
wargame  scenarios. 
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